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ABSTRACT 

The findings, conclusions, and recommendations of the 
Wilms study leave much to be desired. The reported results and 
interpretation of those results are generally not supported, nor was 
the design adequate either to confirm or refute the author* s central 
hypothesis* In its present state, the report of the study cannot 
serve as a valid professional reporting source. The report departs 
from the generally accepted format of scientific reports and mixes 
facts and interpretations rather freely. In order to assess the 
study, therefore, the reviewers were forced to reconstruct it 
according to the logical sequence found in most technical reports, 
i.e., problem, objectives, method, results, conclusions, and 
recommendations. The review conforms to that sequence. The reviewers 
believe that few verifiable results have emerged, or could have 
emerged, from the study because of numerous design flaws and a severe 
lack of the design and procedural information which would be needed 
to evaluate the study's adequacy. Editorializing of the author's 
biases and assumptions occur throughout the report, unsupported by 
study data. When the data do not support the author* s expectations, 
there seems to be a tendency to dispell undesired conclusions with an 
alternative explanation or assumption not based on study data. 
(Author/PR) 
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A BRIEF REVIEW OP WELLFORD W. WILMS' STUDY , 
"THE EFFECTIVENESS OF PUBLIC AND PROPRIETARY OCCUPATIONAL TRAINING" 

The findings, conclusions, and recommendations of this study leave much 
to be desired. The reported results and the interpretation of those results 
are generally not supported, nor was the design adequate either to confirm 
or refute the author's central hypothesis. In its present state, the report 
of the study cannot serve as a source of valid professional reporting. 

It is unfortunate that periodicals such as The Washington Post and the 
Education Daily have not only publicized the report uncritically, but have 
headlined implications which go even beyond the questionable findings stated 
in the report. 

The report departs from the generally accepted format of scientific 
reports and mixes facts and interpretations rather freely. In order to assess 
the study, therefore, the reviewers were forced to reconstruct it according to 
the logical sequence found in most technical reports, i.e., problem, objectives, 
method, results, conclusions, and recommendations. The review which follows 
conforms to that sequence. 

PROBLEM 

Wilms apparently believes that the effectiveness of proprietary and 
public postsecondary occupational training is largely unknown. He does not 
say what he means by "effectiveness." One would suppose that this term 
implies the extent to which the students acquire the skills that the programs 
are designed to teach, but it is not clear that this is what the author has 
in mind. Even more confusing, the author uses the terms occupational 
training , vocational education , and career education interchangeably, giving 
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no indication that he is aware of any distinctions among them. 

OBJECTIVES 

The author gives the objectives of the study in a single paragraph. The 

stated objective of the study was to assess the relative success in the job 

market of graduates of public versus proprietary occupational training 

schools. As was the case with effectiveness, the author does not say What 

he means by success. No operational criteria of success are stated, thus 

making it impossible to either confirm or reject the author's hypothesis that 

After controlling for differences in students' backgrounds 
and abilities, graduates of proprietary schools will 
experience greater occupational success in the labor 
market than graduates of comparable public programs. 

Further reading of the report indicates that the author did use a 
variety of measures of success, but often used them inconsistently. In 
the end, success seemed to be Wilms' subjective impression based upon a 
variety of measures selected a_ posteriori for different situations. The 
measures used were (a) income earned, (b) whether the graduate was working 
in a job considered to be within the occupation for which he was trained, 
(c) various personal characteristics and attitudes, and (d) other mis- 
cellaneous measures. 

METHODS 

Sampling of Schools 

Fifty "occupational training" schools were randomly selected from four 
major U. S. cities, resulting in twenty-one public and twenty-nine proprietary* 
schools. The sample seems to be a heterogenous mix of various kinds of 
schools. Aside from the breakdown into public and proprietary schools, no 
information is provided about the distribution of schools on such important 
variables as location, accreditation, size, type of student, quality of 



-3- 

teachers, or programs offered* Without this infonnation, it is impossible 
to know what universe of schools the sample represents and to what kinds of 
schools the results apply . Further/ the heterogeneity of the sample could 
mask results which might be quite different for different kinds of schools . 
Sampling of Subjects 

Three samples of subjects were selected: beginning students, graduating 
students (students about to graduate) , and graduates (persons who had already 
graduated and were in the labor force)* The author does not state how the 
subjects were selected* It appears as if all the persons in each of six 
programs at the fifty schools were used as subjects, but this is not certain* 
The distribution of subjects across location/ type of school/ and occupation 
is very uneven (Tables 2-4). 

This uneven distribution has resulted in confounding type of school/ 
geographic location/ and occupation* In addition, about fifty percent of 
the graduate sample appear to be the same persons used in the graduating 
sample, resulting in a further confounding. This confoundirfg makes it 
impossible to know whether the results were attributable to real differences 
between proprietary and public schools/ or to artifacts of geographic 
location and occupational differences. 
Sampling of Occupations 

The author selected six occupations of somewhat different prestige 
levels for the study. 
Design and Procedures 

The author does not state what the design was or what procedures were 
used/ except that questionnaires were administered and interviews were 
conducted. Examination of the results/ however/ implies that the procedure 
involved mainly a comparison of the public and proprietary schools on the 
various items in the questionnaires and interviews. In an inconsistent 

O 

ERLC 



-4- 

fashion, some of the items were used as criteria of success, while others 
were used as individual background variables to be 11 control led" in the 
data analysis ♦ 
Instruments 

With two exceptions, the questionnaires are not described, nor is there 
any indication of how many questionnaires there were. With the exception 
of a few items, the interview content and procedures are not described. 

It appears that one of the key criteria of success was whether the 
subject's job matched his training; hence, one of the interview questions 
asked the subject what his or her job was. It is not clear, however, how it 
was determined whether the graduate's job was considered to be within the 
occupation for which he or she was trained. 
Data Analysis 

Wilms used analysis of variance, t- tests, and chi-square to determine 
the differences between public and proprietary schools on a large number of 
the variables. He reports significance levels, but not degrees of association; 
that is, non-zero differences between public and proprietary schools, but no 
indication of the strength of the differences. Consequently, it cannot 
be determined whether reported findings are sizeable or trivial, only that 
they are "not zero." Insufficient information is provided 'about the details 
of the analyses to be able to evaluate their adequacy. In some cases, 
it appears that the data were improperly scaled for the statistical analysis 
used. 

As pointed out previously, type of school is confounded with geographic 
location and occupation, making the meaning of findings very unclear. Table 4, 
for example, indicates that for the Data Proces. ing Programmer occupation 
seventy-five percent of the proprietary school graduates were in Boston and 
Miami, while eighty percent of the public school graduates were in Chicago. 
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Thus, any alleged differences between public and proprietary schools might 
actually have been differences between Chicago and the east coast cities, 
and not have had anything to do with the schools . Since the criterion of 
"success" is heavily weighted with income and employment, factors known to 
vary geographically, this confounding is crucial to any findings involving 
alleged differences in success. For the subsets of data used in many of the 
analyses , no information is provided about the distribution of important 
variables, such as geographic location, occupation, and others across the 
groups being compared. The possibility of additional confounding cannot 
be rulert out. 

RESULTS 

Beginning and Graduating students 

The reported results for these two samples are merely descriptive 
differences between public and proprietary schools on a large number of 
variables. They are not directly relevant to the objectives of the study. 
Because of the confounding problem mentioned earlier, it cannot be concluded 
that the alleged results are actually attributable to real differences 
between the two types of schools. They may be artifacts of geographic 
location or occupation. 
Graduates 

As admitted by the author, no difference in occupational success were 
found between public and proprietary schools, except in the case of secretaries. 
Not even this is a safe conclusion because of the known geographic confounding 
and because of the undefined criteria of success. 
Summary 

The reviewers believe that few verifiable results have emerged, or could 
have emerged, from this study because of numerous design flaws and a severe 
lack of the design and procedural information which would be needed to 
evaluate the adequacy of the study. 



CONCLUSIONS 

In the Abstract of the report of the s 4 tudy are highlighted such findings 
as (a) proprietary instructional costs were 35 percent less than public 
costs; (b) only 20 percent of the graduates of "professional or technical- 
level training" ever got jobs for which they were trained; (c) graduates 
from "lower-level clerical or service woiker programs," with the exception 
of secretaries, barely earned the federal minimum wage; (d) neither kind 
of school compensated for less-advantaged students' backgrounds; and (e) 
graduates of proprietary schools paid 20 times more than public school 
graduates for their training courses. 

Support for these highlighted findings, however, is clouded by the 
previously indicated dearth of key information about the institutions 
involved, the nature of graduates supplying survey data, and the methodologies 
used to collect such data as well as design and analyze flaws. Selective 
reporting -and presentation of survey results encourage misleading inferences 
and conclusions. And, editorializing of the author's biases and assumptions x 
occur throughout the report, unsupported by the data from this study. Thus, 
it seems necessary to challenge the accuracy, completeness, and representativeness 
of the reported conclusions. 

The "Conclusions and Implications" chapter of the report is organized 
around six questions, from which the emphasized findings are drawn. These 
six questions are used below to consider the merit of the author's conclusions 
relative to each. 

Do Graduates of Public or Proprietary Schools Do Better? 

The author concludes that his central hypothesis was not confirmed. 
"With few exceptions, graduates of public schools had about the same success 
in the labor market as graduates of proprietary schools." in elaborating 
upon selected portions of this issue, the author notes the status level of 
jobs gained by graduates and the earnings of various subgroups of the sample. 



It is from these portions that the Abstract draws the findings to be emphasized* 

The adequacy of the sampling of program graduates and of institutions 
is a key element in establishing the merit of the study's findings . Using 
four of the Great Cities as the source of training programs to be studied, 
the report notes that only part of the schools were accredited, and others 
were not. But nowhere does the report inform the reader how many schools 
of what types were surveyed for each specific occupational area, nor how 
many students of particular backgrounds were included from each school and 
program. Thus, when results are later summarized, it is impossible to 
determine whether comparisons are based on graduates of non-accredited 
institutions , primarily from one school in a city, or any other unusual 
characteristic of the particular subsample. 

At the "professional and technical level" of training, upon which the 
20 percent effective placement figure is based, 67 percent of the graduates 
of proprietary schools came from only one city, Chicago. The number of 
schools represented by these graduates is not reported. In Boston, a 
major employment center for electronic technicians, no graduates were surveyed 
from either type of school. In no instance can the reader determine 
whether any accredited proprietary schools were included for a given occupa- 
tional area. Many forms of postsecondary occupational training schools 
are not sampled at all, including public technical institutes, manpower 
training programs (MDTA/CETA), certificate and associate programs of state 
universities, military technical training schools, and community colleges 
serving rural and smaller communities. About half of the graduates were 
very recently graduated; the remainder had graduated about two years previously. 
No distinction is made in their responses, all being pooled as graduates. 
Such restriction of representation must be carefully specified whenever 
results and conclusions of the study are communicated. 



Graduates of accounting programs (requiring two years or less attendence) 
were apparently recorded as "clerks" or as having other lower- level jobs if 
they worked as bookkeepers or payroll clerks. There is no acknowledgement 
that employers may move new workers through a variety of "lower-level" 
positions, to acquaint them fully with company operations before full 
"accountant" positions are assigned. What Wilms seems to be doing is equating 
completion of an accounting program as full preparation for the occupation 
of accountant. 

Also, there is little apparent awareness that skills imparted by a 
particular training program may be highly relevant for performance in many 
types of jobs. For example, accounting skills might reasonably be sought 
by many young people planning to employ such skills in personal affairs, 
sales occupations, or even eventually in a variety of small business 
enterprises. Thus, it might be well to question the merit of the finding 
that more than 80 percent did not get the jobs for which they were trained, 
but became clerks or took low-paying, unrelated jobs. 

When the data do not support the author's expectations, there seems 
to be a tendency to dispell undesired conclusions with an alternative 
explanation or assumption not based on the study data. This happens, for 
example, when too many of the accounting graduates reported that their 
first job after schooling was related to their training. 
D o Public or Proprietary Graduates Experience More Personal Growth? 

It was concluded that "there were no differences in personal growth 
between public and proprietary graduates for any occupational group." No 
consistent differences of any kind were found to influence personal growth 
in a variety of subgroups. But, in view of the limitations in the samples 
of graduates and schools cited previously in this review, no comparative 
conclusions of any kind could have been substantiated or refuted by the 
data and design. 
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Are Public or Proprietary Graduates More Satisfied? 
« Mixed conclusions resulted across the six occupational areas studied. 

In two occupations the author found lower job satisfaction on the part of 
graduates of proprietary schools. However, 80 percent of the proprietary 
graduates for one of the occupations came from only one city. In the other 
s. cupation, 63 percent came from but a single city, and only two cities 
were represented in the total sample. 

♦ 

I Are Public or Proprietary Programs Compensatory? 

By "compensatory/" the author apparently means that the training results 
in women and minorities getting the same salary as male whites in their 
subsequent employment. The author concludes that they do not. In absolute 
terms they earn less in five of the six occupations studied. Aside from 
the limitations of the sampling, this conclusion fails to recognize whether 
such "discriminatory" employment may have been reduced by the training 
programs. Nor does the design of the study permit any evidence of such 
progress to be detected, there being no comparative control groups to pro- 
vide a basis for showing that the training opportunity indeed may be effective 
in this area. 

Are Certain Kinds of Schools Better than Others? 

No differences were found between public schools with respect to institutional 
or staff characteristics influencing how well their graduates fared on the 
job market. However, some features of proprietary schools were concluded 
as having a differential Impact on the hiring and compensation of graduates. 
The mor-> effective schools apparently were larger and their teachers were 
paid more than the less effective schools. Whether this finding was also 
fully correlated with the accreditation status of the schools is not reported, 
but might be suspected. 
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In Public or Proprietary Training Cheaper for the Student? 

She study concludes that "if the public student were not subsidized by 
taxes and paid the same charge to the school as the proprietary student, 
the total costs to the student in our six occupations would be almost equal." 
But the author proceeds to state other cost features which suggest that 
instructional costs are less fo*r proprietary schools and that proprietary 
school students personally pay a far greater amount for their training. 

With respect to student costs, the attention is directed to what was 
"paid," not what it "cost" the student. Yet the study went to great length 
to identify such costs. The analysis revealed ..at "cost" for proprietary 
graduates was only 1.3 times the cost for public graduates, not 20 tiroes 
as great as the direct student payments of tuition and fees. Neither of 
these figures considers that direct payments to proprietary schools may often 
be compensated by the Veterans Administration, Vocational Rehabilitation, 
Social Security, and a variety of other grants-in-aid to students in 
financial need. 

Cost analysis is highly restricted. Teacher salaries and teaching 
load are the sole bases used for assessing instructional cost per student. 
Costs are ignored for such items as physical plant, maintenance, staff fringe 
benefits, extracurricular facilities and activities, advertising, counseling, 
placement services, curriculum updating, investment risk capital, teaching 
equipment and aids, and personnel and business management operations. 

RECOMMENDATIONS 

When presenting testimony that is likely to spark controversy on important 
public issues, the professional researcher had better be able to describe 
completely the nature and adequacy of his data collection and analysis. 
The study does not do this. Neither the sampling design nor the reported 
results supports conclusions that substantiate the "findings" headlined by the report. 

12 
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Yet from this research experience, Wilms adds his recommendations for making 
postsecondary occupational training schools "more effective institutions 
for a democratic society 

Proceeding from questionable and selected findings, he presents seven 
recommendations for governmental action. The key element of these recommendations 
appears to be the provision of information on how well each school is able 
to place graduates in jobs for which they have been trained. 

Pursuit of these recommendations should be handled with serious concern 
and due caution. If a school's placement record were to become the key 
factor in evaluating its success (hence, its survival) , then it would be 
reasonable to expect that employment opportunities should come under its 
control to sane extent. If not, a school's most likely recourse would be 
to conduct training programs only for the high-volume manpower needs of 
the state. This would seem to dictate a neglect of newly emerging occupa- 
tions, experimental programs, and the individual interests of students, 

A second theme of the recommendations is establishment of equality 
of pay for equal work, such that schools may be "legally culpable when they 
actively engage in job placement and referral activities that are discriminatory. » 
Other than forcing schools to be more selective in admissions, another potential 
result of implementing such a recommendation could be to force schools 
to eliminate placement and referral services for students. Neither 
consequence seems helpful to the target groups of students who may already 
be considered as disadvantaged. 
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DISTRIBUTION OF GRADUATES OF PUBLIC AND PROPRIETARY 
SCHOOLS BY OCCUPATIONAL PROGRAM AND GEOGRAPHIC AREA 





San Fran. 
Bay Area 


Chicago 


Boston 


Miami 


Totals*. i'{ 


Occupation 


Public 


Prop. 


Public 


Prop. 


Public 


Prop. 


Public 


Prop. 






25 


28 


39 


29 


76 


94 








Electronic data pro- 
cessing programmer 






35 


98 


60 


9 


45 


15 


262 


Dental assisting 


98 


174 






105 


104 






481 i 


Electronic technician 






22 


279 






104 


57 


462 


Secretarial 


13 


16 


3 


189 


51 


79 


10 


25 


386 


Cosmetology 


103 


153 










18 


114 


388 


TOTALS 


239 


371 


99 


595 


292 


286 


177 


211 


2270 ; 



15 



